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Summary 

I  For  unbalanced  Model  I  ANOVA  data  there  is  a  controversy 
in  the  literature  --  should  the  partial  SS's  be  computed 
hierarchically  or  should  they  be  computed  after  fitting  all 
other  model  terms  including  higher-order  interactions?  A 
similar  problem  exists  in  the  Model  I  analysis  of  covariance 
(ANCOVA) ,  yet  the  usual  prescription  given  in  textbooks  uses 
only  the  non-hierarchical  approach.  Using  two  classic  examples 
of  ANCOVA,  this  paper  examines  the  two  approaches  to  ANCOVA 

calculations .  . 
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1.  Introduction 


In  recent  years  much  attention  has  been  given  to  the 
question  as  to  what  is  the  correct  method  of  analysis  for 
data  from  an  unbalanced  Model  I  analysis  of  variance  (ANOVA) . 

A  principal  area  of  contention  has  been  whether  the  SS  to  be 
used  in  the  numerator  of  an  F-test  should  be  computed  assuming 

higher-order  interactions  are  zero,  or  whether  it  should  be 
the  partial  sum  of  squares  for  the  effect  under  test  after 

fitting  all  other  model  terms  including  higher  order  inter¬ 
actions.  Both  approaches  have  their  advocates.  Of  course, 
in  the  balanced  ANOVA  case  there  is  no  dispute,  since  the  two 
approaches  are  equivalent  there. 

What  does  not  seem  to  have  been  recognized,  or  at  least 
clearly  exposited  in  the  literature  and  especially  in  textbook 
discussions,  is  that  a  question  exists  as  to  what  is  the  correct 
method  of  analysis  in  the  Model  I  analysis  of  covariance 
(ANCOVA) ,  even  in  the  case  of  a  balanced  design.  Unless  the 
covariate  is  itself  "secretly  balanced",  its  inclusion  in 
the  analysis  automatically  unbalances  a  balanced  design. 

The  choice  of  an  appropriate  method  of  analysis  is  (or  should 
be)  essentially  the  same  for  both  the  unbalanced  Model  I  ANOVA 
and  the  Model  I  ANCOVA.  How  is  the  matter  dealt  with  in  text¬ 
books? 

The  usual  prescription  given  in  textbooks  for  computing 
an  adjusted  sum  of  squares  for  one  out  of  several  sources 
of  variation  follows  the  classic  prescription  of  Cochran  (1957), 
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reproduced  here  in  Table  1,  and  yields: 

SS  »  [E  +T  -CE  +T  )^/(E  +T  )  ]  -  [E  -E^  /E  ] 
adj  yy  yy  xy  xy'  '  ^  xx  xx’’ ^  yy  xy'  xx 

■  T  +  E^  /E  -  CE  +T  )^/(E  +T  ). 
yy  xy'  XX  '■  xy  xy*  '  ^  xx  xx' 


(1) 


The  first  form  of  is  in  the  form  of  a  difference  of  two 

residual  SS,  one  excluding  the  effect,  the  other  including  it.  It 
reveals  that  SS^^j  is  in  fact  a  partial  SS  for  the  effect, 
after  fitting  all  of  the  other  terms  in  the  unadjusted  ANOVA 
including  the  corresponding  interaction  terms,  i.e.,  the 
second  approach  mentioned  above.  In  this  paper  we  refer  to 
an  analysis  based  on  such  a  prescription  as  the  "textbook" 
analysis . 

--  Table  1  goes  about  here  -- 
If  we  were  to  write  out  the  ANCOVA  model  for  a  design 
with  at  least  two  dimensions  (sources  of  variation)  as  a 
regression  model,  omitting  design  variables  corresponding  to 
interaction  terms,  and  then  compute  the  SS  for  a  specific 
source  of  variation,  we  would  typically  get  a  different 
answer  --  one  which  corresponds  to  assuming  that  the  higher- 
order  interaction  terms  are  zero,  i.e.,  the  first  approach 
mentioned  above.  Here,  we  shall  refer  to  such  a  SS  calcula¬ 
tion  as  being  "hierarchical",  as  contrasted  with  the  non- 
hierarchical  nature  of  the  "textbook"  analysis.  The  hier¬ 
archical  ANCOVA  approach  is  notable  for  its  absence  from 
the  major  textbooks  dealing  with  experimental  design.  However, 
in  the  balanced  case  it  is  no  more  computationally  complex 
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than  the  textbook  ANCOVA  and  in  the  unbalanced  case  may  in 
fact  be  simpler.  The  adjusted  SS  for  a  main  effect  (effect 
lower  in  hierarchy  than  interactions)  is 


fE  +I  +T 
l  yy  yy  yy 


i4x“xx5'’ 


(E  +I 

.p  .  xy  xy-' 
^yy  (E  + 1  '  J 

'•XX  XX'' 


(E  ■►I  '•■T  V 

'•  xy  xy  xy'' 

■  (E  ■  •►1  -t-T  ) 

'•  XX  XX  XX 


(2) 


where  I  ,  I  ,  I  are  sums  of  interaction  SS's  (terms  higher 
7/  *y 

in  hierarchy  than  1^^.,  etc.) 

In  Section  2  we  illustrate  the  two  solutions  to  the  ANCOVA 
calculations  using  two  classic  examples,  one  taken  from  the 
2nd  edition  of  the  text  by  Cochran  and  Cox  (1957),  and  the 
other  (an  unbalanced  design)  from  Federer  (1957),  in  the 
special  Biometrics  issue  on  ANCOVA.  In  Section  3  we  comment 
on  these  and  other  textbook  examples  we  have  examined,  make 
some  related  points  regarding  assumptions  for  ANCOVA  calcula¬ 
tions,  and  argue  that  those  who  oppose  the  non-hierarchical 
approach  to  unbalanced  ANOVA,  should  also  eschew  it  in  the 
ANCOVA. 
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2.  Two  Examples 

We  begin  with  an  example  from  Federer  (1957),  the  data 
for  which  are  included  here  in  Table  2.  The  structure  here 
is  actually  a  one-way  design  with  5  treatments  (levels)  in 
randomized  blocks  with  \anequal  replication  within  blocks. 
Federer 's  analysis  treats  this  as  a  two-way  design,  keeping 
the  block  x  treatment  interaction  separate  from  the  within 
block  error  and  implicitly  assuming  the  unweighted  sums  of 
the  block  effects  and  of  the  treatment  effects  are  zero  in 
defining  interactions.  Federer  does  not,  however,  test  for 
blocks,  and  thus  recognizes  their  special  role.  Our  analysis 
parallels  Federer 's  in  both  these  regards. 

--  Table  2  goes  about  here  -- 

Federer 's  analysis  is  notable  because  he  includes  all  of 
the  SS's  necessary  to  dq  both  the  textbook  and  hierarchical 
analyses  in  his  Table  10,  but  he  deliberately  chooses  to  use 
the  "textbook"  analysis,  testing  treatment  effects  after 
eliminating  blocks,  interactions ,  and  the  covariate.  The  left- 
hand  panel  of  Table  3  contains  the  relevant  SS's  from  Federer 
(slightly  corrected),  and  the  middle  panel  contains  his  "text¬ 
book"  analysis  of  treatment  effects.  His  calculations  yield 


(T) 


SS(TlB,TxB,x) 


1606.67  + 


(6.00)^ 

TT'OO 


(367.33+6.00)^ 

T172:!'3+4.o6) 


-  826.14. 


The  right-hand  panel  of  Table  3  contains  the  "hierarchi¬ 


cal"  analysis  for  which 


s 


SS (TlB,x) 


1320.96  + 


(139.80+6.00)^ 

(89.47+4.00) 


(303.81+139.80+6.00)^ 

(136.12+^.47+4.00) 


»  667. 91. 


(As  we  would  expect  the  line  for  the  interaction  term  is  the 
same  in  both  analyses.)  The  F-ratio  for  treatments  in  the 
"textbook"  approach  is  3.60  which  falls  just  below  F  jq(4,4)  ■ 
4.11,  while  the  F  value  in  the  "hierarchical"  analysis,  2.91, 
is  closer  to  F  25(4,4)  »  2.06,  than  to  the  0.10  value.  Here, 
the  two  solutions  differ  sufficiently  to  make  a  difference 
in  the  conclusions  that  might  be  drawn. 

--  Table  3  goes  about  here  -- 
Our  second  example  is  the  primary  illustration  of  ANCOVA 
in  Cochran  and  Cox  (1957).  The  data  are  given  there  in  Table 
3.1,  p.  46,  and  come  from  an  experiment  in  4  blocks  of  12 
plots  each.  Four  fumigants  were  used  at  dosage  levels  0,  1, 
and  2  in  each  block,  and  thus  in  each  block  we  have  4  controls, 
1  plot  of  each  fumigant  at  level  1,  and  1  plot  of  each  at 
level  2.  The  model  used  in  Cochran  and  Cox's  ANOVA  analysis 
in  Table  3.5  can  be  written  as 

^ijk  “  W  ^ii(j"^^ 

*  (3) 


where 


is  the  block  effect,  i  indexes  the  fumigants,  j  indexes 
level ,  and 


^11  *  ^12 


(4) 


for  1,2.  The  response  y  is  the  number  of  eelworm  cysts 
in  a  sample  of  400  grams  of  soil.  The  covariate  used  by 
Cochran  and  Cox  in  their  analysis  of  covariance  on  pp.  84-86 
is  the  count  for  a  sample  of  400  grams  of  soil  before  fumiga¬ 
tion. 


What  makes  this  example  of  some  interest  is  that  Cochran 
and  Cox  use  orthogonal  polynomial  contrasts  to  partition  the 
treatment  SS  appropriately  for  the  hypotheses  of  interest. 
These  are  implicitly  determined  in  a  particular  order  in  which 
(j-1)  is  made  orthogonal  to  the  constant  and  (j-l)^  is  made 
orthogonal  to  the  constant  and  (j-1).  Given  these  contrasts 
there  is  no  dispute  as  to  how  one  should  compute  the  ANOVA 
table  corresponding  to  the  model  in  expression  (3).  Once  we 
add  the  covariate  the  model  becomes 


^ijk  ■  “  *  *  6jj(j-l) 

«  -  il  .  tS) 

and  one  would  hope  that  the  ANCOVA  adjusted  SS's  retain  the 
orthogonality  of  the  original  analysis.  Sadly,  the  introduc¬ 
tion  of  the  covariate  destroys  the  orthogonality. 

In  the  left-hand  panel  of  Table  4  we  give  the  basic 
sura  of  squares  and  products  from  Cochran  and  Cox's  Table  3.9; 
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in  the  middle  panel  we  give  their  "textbook"  analysis  values 
of  SS^dj  f  for  each  line.  Finally,  in  the  right-hand 
panel  we  give  the  hierarchical  analysis. 

--  Table  4  goes  about  here  -- 

The  two  analyses  in  this  example  would  likely  lead  to 
the  same  conclusions  --  the  linear  and  linear  x  fumigant  effects 
seem  important  ("significant"),  but  the  quadratic  and  quadratic 
X  fumigant  effects  are  much  less  so.  The  calculations  do 
differ,  however,  and  in  some  settings  the  difference  between 
an  F-value  of  1.47  and  one  of  1.88  (with  1  and  35  d.f.)  might 
matter  to  someone. 

We  find  this  second  example  especially  instructive  because 
it  serves  to  remind  us  that  "nice"  single  degree  of  freedom 
contrasts  and  interactions  in  an  ANOVA  may  become  nontrivial 
to  deal  with  if  we  wish  to  adjust  for  a  covariate. 
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3.  Discussion 

In  the  previous  two  sections  we  have  noted  that  the 
controversy  surrounding  the  calculation  of  SS's  in  unbalanced 
Model  I  ANOVA  should  carry  over  to  Model  I  ANCOVA,  even  in 
the  balanced  case.  Yet  among  those  textbooks  that  we  have 
examined  (e.g.,  Bliss,  1970;  Rao,  1973;  Steel  and  Torrie,  1960) 
which  include  a  presentation  of  analysis  of  covariance  beyond 
the  simple  one-way  case,  we  have  found  only  examples  of  the 
non-hierarchical  approach  which  use  the  formula  for 
given  in  expression  (1). 

We  find  the  dominance  of  this  non-hierarchical  approach 
to  ANCOVA  in  textbooks  puzzling,  because  many  of  these  books 
adopt  (or  at  least  discuss)  hierarchical  analyses  in  Model  I 
ANOVA.  Ther6  is  a  sense  in  which  this  dominance  should  be 
surprising  because  the  ANCOVA  model  is  itself  the  product  of 
"hierarchical  thinking".  The  basic  ANCOVA  model  includes 
only  a  single  regression  coefficient  for  the  covariate,  and 
this  assumes  the  absence  of  treatment  x  covariate  interactions. 
Indeed,  it  is  via  the  simple  1  d.f.  adjustment  that  ANCOVA 
gains  its  strength.  A  completely  consistent  non-hierarchical 
approach  to  ANCOVA  would  adjust  for  treatment  x  covariate 
interactions  as  well.  This  would  differ  from  both  the  "text¬ 
book"  and  the  "hierarchical"  approaches  described  here. 

How  much  of  a  difference  might  we  expect  to  find  between 
the  "textbook"  and  "hierarchical"  approaches?  In  those 
examples  which  we  have  examined  the  difference  between  them 


is  usually  not  great.  We  have  found  examples  (e.g..  Bliss,  1970 
pp.  SlOff)  in  which  the  analysis  differ  drastically,  but 
they  have  exhibited  substantial  interaction,  in  which  case 
the  hierarchical  main  effect  test  is  meaningless.  The  two 
examples  in  Section  2  are  as  dramatic  as  any  others  we  have 
encountered  where  interaction  is  not  present.  Yet,  such 
differences  are  of  the  same  order  of  magnitude  as  those 
between  the  "textbook"  F- values,  and  those  calculated  by 
doing  an  ANOVA  of  y  -  yx  (see  Cochran  and  Cox,  1957;  p.  87). 
Table  5  lists  the  3  sets  of  F-values  for  their  example 
analyzed  here  in  Table  4.  If  we  go  to  the  trouble  of  doing 

A 

the  "correct"  analysis,  instead  of  doing  an  ANOVA  on  y  -  yx, 
then  perhaps  we  should  worry  a  little  about  whether  we  should 
do  the  "textbook"  or  the  "hierarchical"  analysis. 

Although  we  have  not  carefully  examined  the  standard 
computing  packages  to  see  how  they  handle  this  problem,  it 
is  our  impression  that  most  of  the  detailed  ANCOVA  programs 
for  balanced  layouts  use  the  "textbook"  formula,  while  the 
more  general  ANCOVA  programs  do  a  regression-like  approach. 

In  the  latter  case  ANCOVA  output  will  be  of  the  textbook 
(non-hierarchical)  or  hierarchical  form  depending  on  which 
approach  the  package  uses  for  its  regression  ANOVA  table. 

Since  there  are  some  programs  that  report  both  hierarchical 
and  non-hierarchical  SS’s,  they  have  the  ability  to  produce 
both  types  of  ANCOVA  calculations. 


--  Table  S  goes  about  here  -- 

Our  purpose  here  has  not  been  to  advocate  the  hierarchi¬ 
cal  approach  to  ANCOVA  calculations,  although  it  is  the  one 
we  favor.  Rather  we  have  tried  to  point  out  what  we  believe 
to  be  an  anomaly  in  the  standard  textbook  treatment  of  ANCOVA 
that  disguises  the  fact  that  there  should  be  some  controversy 
over  what  is  the  correct  method  in  the  Model  I  ANCOVA,  even 
in  the  balanced  case.  Thus,  those  who  oppose  the  non-hier- 
archical  approach  in  the  case  of  unbalanced  ANOVA,  should 
also  eschew  it  in  the  analysis  of  covariance. 


Table  1 


(a)  Table  1  from  Cochran  (1957)  Showing  Sums  of  Squares 
and  Products  for  ANCOVA 


D.f. 

(*») 

V) 

TraatmeBti 

«-l) 

T„ 

Error 

/. 

Sum 

(b)  Table  2  from  Cochran  (1957)  Showing  Partition  of  E 
and  S  Into  Components 


(»•) 

Riyomion 

DeriotioM 

D.  f.  8. «. 

D.f. 

an  M.a 

Error  Em 

1  EU/Em 

/.-I 

a.  -  0&.  •: 

Sum  Sm 

1  .  Si,/Sm 

t+f.- 2 

S„  -  S’m/Sm 

Trmtmeota  (by  wbtraetion) 

t  -  1 

T„  -  Sl,/Sm  +  SU/Em  *! 
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Table 


2.  Example  from  Federer  (1957;  Table  9,  p.  356) 
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of  Squares  and  Products  I  I  Textbook  Analysisl  Hierarchical  Analysis 


Table  4.  Analysis  of  Covariance  for  Data  in  Cochran  and  Cox  (1957),  Table  3.1,  Including 
Values  Reported  in  Tables  3.9  and  3.12 


Table  5.  F-Values  for  Components  in  Table  4 


F'Values 

Source  of  Variation 

df 

ANOVA 

on 

y  -  Yx 

Textbook 

Hierarchical 

Linear 

1 

14.88 

14.51 

13.37 

Quadratic 

1 

1.50 

1.47 

1.88 

Differences 

in  Linear 

3 

5.64 

5.05 

5.08 

Differences 

in  Quadratic 

3 

0.92 

0.92 

0.92 

Error 
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